Reliability and Generalization Studies Reliability: Arguments for Multiple Perspectives and Potential Problems with Generalization across Studies
نویسنده
چکیده
The present article addresses reliability issues in light of recent studies and debates focused on psychometrics versus datametrics terminology and reliability generalization (RG) introduced by Vacha-Haase. The purpose here was not to moderate arguments presented in these debates but to discuss multiple perspectives on score reliability and how they may affect research practice, editorial policies, and RG across studies. Issues of classical error variance and reliability are discussed across models of classical test theory, generalizability theory, and item response theory. Potential problems with RG across studies are discussed in relation to different types of reliability, different test forms, different number of items, misspecifications, and confounding independent variables in a single RG analysis.
منابع مشابه
The Hamilton Rating Scale for Depression: A meta-analytic reliability generalization study
Reliability generalization is a meta-analytic approach to study how reliability estimates of a test scores depend on the specific characteristics under which the test is applied and, as a consequence, the risks of inducing score reliability from previous applications of the test. The Hamilton Rating Scale for Depression (HAMD) is one the most popular measurement instruments in clinical psycholo...
متن کاملMeta-analysis of coefficient alpha.
The meta-analysis of coefficient alpha across many studies is becoming more common in psychology by a methodology labeled reliability generalization. Existing reliability generalization studies have not used the sampling distribution of coefficient alpha for precision weighting and other common meta-analytic procedures. A framework is provided for a statistically grounded meta-analysis of coeff...
متن کاملMyers-briggs Type Indicator Score Reliability across Studies: a Meta-analytic Reliability Generalization Study
متن کامل
Effects of the heterogeneity of the variances on reliability generalization: an example with the Beck Depression Inventory.
The studies of Reliability Generalization (RG) analyze estimates of the reliability of scores from a test provided by a set of studies. As their goals and designs are usually very varied, the sampling of individuals obeys very different schemas. Thus, the variances of the scores might be more heterogeneous than expected from random sampling. Two main problems associated with this potential sour...
متن کاملReliability generalization: lapsus linguae
RELIABILITY GENERALIZATION: LAPSUS LINGUAEby JULIE M. SMITHDecember 2011Advisor: Dr. Shlomo S. SawilowskyMajor: Education Evaluation and ResearchDegree: Doctor of PhilosophyThis study examines the proposed Reliability Generalization (RG) method for studyingreliability. RG employs the application of meta-analytic techniques similar to those used invalidity gen...
متن کامل